Enhancing Interaction with Dual-Screen Television Through Display Commonalities

Second screening - engaging with a mobile device while watching TV - is ubiquitous. Previous research demonstrates that this is hampered by cognitive and physical disjuncts between the simultaneous content streams. To engage effectively with more than one screen, users must manage their attention, for example, by frequently adjusting their gaze or posture. This can lead to cognitive effort, which leads to disengagement, content sacrifice, and ultimately, affects user experience (UX) negatively. In this paper, we look to improve the design of the dual-screen scenario through \emph{display commonalities}; the mirroring of one content stream (e.g., TV material or second screen content) within the other. We evaluate this design space with professional broadcast practitioners, and then conduct an empirical investigation to determine the impact of the most successful methods towards understanding their impact, and designing towards positive UX with multi-device scenarios

Understanding and Designing Attention for Dual-Screen Media

Recently there has been a transformative shift towards engaging with mobile devices while watching television. Content creators, therefore, wish to create applications to support these behaviours to provide more engaging multi-device TV. Currently, their designs do not reflect the subtle variations in viewer attention, our physiological capabilities, or the additional mental effort such scenarios imply. We investigate this in two primary ways: by further understanding the current issues faced by users when dual-screening, and by designing a series of technological interventions for managing cross-device attention. First, we conduct two studies to better understand the user experience of second screening. Through a large-scale online questionnaire and a series of interviews we document the problems faced by users when second screening and how they compensate and mitigate for missing content when engaging with mobile devices. We then conduct an investigation to explore the effect of dual-screen visual complexity in terms of objective and subjective experience of participants when exposed to content of varying complexity across two screens.For our technological interventions, we first investigate how visual complexity on a mobile device may be varied to account for the perceived complexity of TV material by loading textual material at varying levels of complexity. We explore the tradeoff of user autonomy and content creator control by contrasting the effects of users adjusting the complexity themselves, and automatic adjustment around heuristics. Then, we consider how different audio-visual stimuli may be used to direct a user’s attention between screens at key moments. Finally, we explore how we can support users to reduce the switching costs and cognitive effort associated with engaging with cross-device media mirroring unattended visual information in the experience on an attended screen. Throughout the thesis we show that many of our interventions are a beneficial state of the art and form a series of guidelines for each. The thesis concludes by offering an outline of our contributions and a framework for others to extend our work

JDLED: towards visio-tactile displays based on electrochemical locomotion of liquid-metal Janus droplets

An actuated shape-changing interface with fast response and small pixel size using a liquid material can provide real time tangible interaction with the digital world in physical space. To this end, we demonstrate an interface that displays userdefined patterns dynamically using liquid metal droplets as programmable micro robots on a flat surface. We built a prototype using an array of embedded electrodes and a switching circuit to control the jump of the droplets from electrode to electrode. The actuation and dynamics of the droplets under the finger provides mild tactile feedback to the user. Our demo is the first to show a planar visio-tactile display using liquid metal, and is a first step to make shape-changing physical ephemeral widgets on a tabletop interface

Programmable liquid matter: 2D shape drawing of liquid metals by dynamic electric field

We present a programmable liquid matter which can dynamically transform its 2D shape into a variety of forms and present unique organic animations based on spatio-temporally controlled electric fields. We deployed a EGaIn (Gallium indium eutectic alloy) liquid metal as our smart liquid material since it features a superior electric conductivity in spite of a liquid state and presents a high dynamic range of surface tension and 2D area controlled by applied voltage strength and polarity. Our proposed liquid metal shape and motion control algorithms with dynamically patterned electric fields realize path tracing organic animation. We demonstrate an interactive 7x7 electrode array control system with a computer vision based GUI system to enable novice users to physically draw alphabet letters and 2D shapes by unique animatronics of liquid metals